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ABSTRACT 

In 1991-92 Vermont initiated a statewide portfolio assessment 
program in mathematics and writing in grades 4 and 8. An evaluation of 
the program found mixed results although the portfolio scores were 
unreliable for individual or school-level reporting and teachers had 
concerns about implementation, the assessment had marked positive 
effects on instruction. This paper describes the implementation and 
impact of the second year of the assessment in 1992-93. Issues 
addresred include time spent on mathematics portfolios and best pieces, 
inservice training and support, changes in curriculum and instruction, 
portfolio preparation and scoring practices, teacher and student 
attitudes toward the portfolios and student performance. 
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INTRODUCTION 



For the past five years Vermont has been developing an innovative 
statewide assessment system in which portfolios of students' work in 
mathematics and writing are a key element. The Vermont program has two 
primary goals --to provide a valid measure of student performance and 
to encourage changes in curriculum and instruction that promote higher- 
order thinking. Vermont is the first state to make portfolios the 
centerpiece of a statewide assessment system, and the state's pioneering 
efforts have attracted nationwide attention. The Vermont experience 
provides a valuable source of information about the challenges and 
consequences of using portfolios in large scale assessment and using 
assessment to drive curriculum reform. 

Since 1990, the year portfolios were first piloted, RAND has been 
evaluating the program under the auspices, of the Center for Research on 
Evaluation, Standards, and Student Testing (CRESST) . This paper 
describes the implementation and impact of mathematics portfolios in the 
second year of statewide implementation, 1992-93. This evaluation 
focuses primarily on the mathematics portfolio assessment because the 
use of portfolios in this subject represents a clearer break from extant 
practice and is, from a national perspective, more unusual. 

Many of the issues raised in RAND's evaluation of the pilot year 
and first year of statewide implementation (Koretz et al., 1993) are 
revisited here. Last year's teacher survey revealed significant changes 
in curriculum and instructional practices consistent with the goals of 
the portfolio assessment, but also indicated that portfolios placed a 
substantial burden on teachers' time. Moreover, there were significant 
variations in teachers' approaches to portfolios, which would affect the 
interpretation of portfolio scores. This year's results are similarly 
mixed. While desired changes in curriculum and instructional focus were 
sustained in the second year, there was no substantial reduction in the 
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time burden portfolios placed on teachers. About one-half of the 
teachers see a payoff for their extra effort in terms of improved 
student learning; one-half do not. Substantial variations in teachers' 
approaches to portfolios also persist. Finally, although most teachers 
express continued support for math portfolios, they have strong 
reservations about some of the state's goals for the program, 
particularly the emphasis on using portfolio results for formal external 
assessment purposes . 

PROCEDURES 

This study is based on data from questionnaires that were 
distributed to all teachers of mathematics in grades 4 and 8 in the 
spring of 1993 along with the state's Uniform Test. 1 Teachers were 
^asked to complete the questionnaires anonymously and to return them with 
the completed student test booklets. Most survey questions were Likert- 
type items, requiring respondents to select one of five or six ordered 
responses. A few items required teachers to estimate the percentage of 
time devoted to particular activities or the percentage of students 
behaving in certain ways. There were two open-ended items requiring 
written responses. 

Fave hundred nineteen completed questionnaires were returned, 
three- fourths from grade 4 and one-fourth from grade 8. This represents 
approximately 52 percent of all Vermont teachers who taught mathematics 
in grade 4 and 41 percent of mathematics teachers in grade 8.^ Although 
this response rate is much lower than last year's (83%), the total 
number of respondents is more than three times as large, including one- 
half of the entire population. Responses to all survey items were 
keypunched and analyzed by computer. A random sample of 50 percent of 

: This is the third year in which teacher questionnaires have been 
administered as part of the RAND evaluation; however, previous 
questionnaires were sent only to a sample of teachers. 

; - These are the most conservative estimates of the response rate. 
They are based on the total number of teachers who teach mathematics at 
each grade level in the state. However, because of variations in 
distribution and testing procedures at the local I 3vel , we do not know 
that all eligible teachers received the survey. 
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the papers with written responses to open-ended items was selected, and 
these responses were read, summarized, and tabulated by hand. 



This section begins with a description of the characteristics of 
teachers who completed the survey, then proceeds with a thematically 
organized discussion of the research findings. This discussion focuses 
on questions related to: implementation (specifically, inservice 
training and portfolio practices at the classroom level); impact 
(changes in curriculum and instruction, student performance, and teacher 
attitudes); and the burdens portfolios place on teachers and students. 

Teacher Characteristics 

The characteristics of teachers who responded to the survey are 
almost identical to those of the 1991-92 random sample, giving us more 
confidence in the generalizability of the survey results. The typical 
Vermont mathematics teacher has considerable classroom experience. (see 
Table 1) On average, eighth grade teachers have 16 years of experience 
and fourth grade teachers a little less than 15 years. Less than 10 
percent: of the- respondents have under four years of experience. 
Consistent with traditional elementary and middle school scheduling 
practices, 70 percent of eighth grade teachers specialize in teaching 
mathematics (as opposed to teaching many subjects) while less than two 
percent of fourth grade teachers specialize in mathematics'. 



A large majority (82%) of respondent" have at least one year's 
previous experience with the mathematics portfolios. For the most part, 
those teachers (18%) who had not used math portfolios before the 1992-93 
school year simply did not teach mathematics at grade 4 or 8 in 1991-92. 
Over 2C percent of the teachers had two years experience with the math 
portfolio: one-fifth of the fourth grade teachers and one-t./rd of the 
eighth grade teachers participated in the 1990-91 pilot program. 



RESULTS 



Insert Table 1 about here 
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Training and Support 

Vermont has provided portfolio-related training activities each 
year to meet the needs they perceived to be the greatest. Fcr example, 
during the first year of implementation the focus of training was on 
explaining portfolio procedures and finding appropriate tasks. In 1992- 
93, the training focused on the scoring criteria. The level of 
satisfaction with training in 1992-93 was comparable to 1991-92. Over 
one-half of the teachers at bcch grade levels feel adequately prepared 
to work with the mathematics portfolios as a result of the training they 
received. Shortcomings in training were reported more often at grade 4 
than grade 8. One-quarter of the fourth grade teachers, compared to 
only 12 percent of the eighth grade teachers, feel poorly or very poorly 
prepared to work with the portfolios. Fourth grade teachers also rate 
the network scoring training sessions somewhat lower than eighth grade 
teachers. Approximately one-half of eighth grade teachers rate the two 
network scoring training sessions as good or very good, compared to 
about 40 percent of the fourth grade teachers. 

Less than 10 percent of teachers wrote open-ended comments 
specifically about training. Most of these teachers say that training 
sessions placed too much emphasis on scoring portfolios and not enough 
atcention was given to how to teach portfolios effectively. A few 
teachers complain of having to be away from their students too often to 
attend training sessions. 

Variations in Classroom Implementation of Portfolios 

One concern raised loudly in last year's RAND evaluation and echoed 
in teacher's open-ended comments this year was that portfolios are not 
implemented uniformly across classrooms and schools. Several items on 
the questionnaire reveal extensive variation in portfolio-related 
policies and practices. For the most part, this variation has not 
lessened since 1991-92. The percentage of teachers reporting that 
students' generally revise their best pieces at least once has risen 
from 7 3 to 8G percent. The mean number of revisions at grade 4 is 
virtually unchanged from last year (1.17), but there is a modest 
increase from 1.00 to 1.10 at grade eight. Nevertheless, teachers' 
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policies on revising best pieces still vary significantly. . Although 57 
percent of teachers encourage revision of most best pieces and 19 
percent permit revision, another 19 percent require at least some 
revision, and 5 percent generally do not per/nit revisions. Similarly, 
the amount of time students spend revising varies widely. The average 
revising time is 30-40 minutes, but in roughly 17 percent of classrooms 
students do not revise at all. In another 15 percent of classrooms 
students take more than one full class period to revise a best piece. 
Students who are not encouraged or allowed to revise their best pieces 
will clearly be at a disadvantage relative to those who are encouraged, 
or even required to revise their work. 

Teachers also set policies regarding who may assist students in 
revising their best pieces. One in four teachers do not assist their 
own students in revisions, and a similar proportion do not permit 
students to help each other. Seventy percent of fourth grade teachers 
and 39 percent of eighth grade teachers forbid parental or other outside 
assistance, (see Table 2) The remaining teachers permit their students 
to receive outside assistance. This is consistent with 1991-9? results 
which indicated that 65 percent of teachers at grade 4 and 43 percent at 
grade 8 placed some limit on parental assistance with portfolio 
projects. Further complicating matters, roughly 10 percent of teachers 
have different rules for each student. Teachers' policies also differ 
with respect to acknowledgment of outside help. Only about 20 percent 
require students to acknowledge or describe the assistance they receive. 



Insert Table 2 about here 



The type and quality of the work that becomes pare of a students' 
portfolio is also heavily influenced by teachers' decisions about how 
best pieces are selected. Fourth grade teachers generally provide 
students with more guidance in selecting best pieces than do eighth 
grade teachers. However, this year's survey, like last year's, reveals 
substantial differences in the amount of teacher influence within grade 
levels, with some teachers playing an equal role with the student and 
others playing no role at all. {see Table 3) 
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Insert Table 3 about here 



On the other hand, there are many similarities in portfolio 
practices. Most teachers are using mathematics portfolios with nearly 
all of their students: ninety-six percent reported that most, almost 
all or all of their students are compiling mathematics portfolios. 
Those who are excluded are primarily students from other grade levels 
who are enrolled in multi-grade classes. About 15 percent of teachers 
also excuse some special education students from participation in the 
portfolio assessment. 

Another area of congruity is in teacher's decisions about how much 
emphasis to place on different characteristics of best pieces. These 
decisons can have a subtle, but systematic, influence on the types of 
work students include in their portfolios. The vast majority of 
teachers place a moderate or heavy emphasis on the assessment scoring 
criteria, and also on work that is "interesting or important to 
students." Most teachers place minor or moderate emphasis on students 
r.ioces being mathematically correct and having a neat and polished 
appearance, (see Table 4) Other than a small decrease in emphasis on 
student work being similar to examples in the Resource Guide, there has 
been very little change in emphasis since last year. It may be that 
training and scoring experience have helped to bring about this 
consistency of approach. 



Inser* able 4 about here 



Changes in Curriculum and Instruction 

One of the major goals of the mathematics portfolio program is to 
improve curriculum and instruction at the classroom level. In an 
attempt to measure these changes, we asked teachers to compare their 
current teaching activities with their approach before they started 
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using portfolios. 3 As in 1991-92, most teachers report substantial 
changes in curriculum focus and teaching methods* since they began using 
portfolios, changes that are consistent with the goals of the assessment 
program. These changes are more pronounced in the fourth grade than the 
eighth grade, which may be attributed to greater flexibility in 
scheduling and curriculum. 

Curriculum changes are greatest in the areas of probleir- solving and 
mathematical communication, which are emphasized by the Vermont 
mathematics portfolio assessment. Most teachers are spending more 
classroom time in these areas in 1992-93 than they did prior to using 
portfolios. In the fourth grade 83 percent of teachers devote more 
class time to "learning problem solving techniques" than they did before 
the introduction of mathematics portfolios. (see Table 5) Over 70 
percent of fourth grade teachers say they spend more class time applying 
math to novel and real world problems and solving logic or reasoning 
problems. Of the eight specific problem solving activities mentioned in 
the survey, only traditional v/ord problems are receiving the same or 
less class time in more than one-half of fourth grade classes. 



Insert Table S about here 



r:\iriicular and instructional changes are not as qreat in the eight.,! 
arade. Fewer eighth grade teachers report increases in class time 
devoted to problem solving than fourth grade '-eachers. Less than one- 
halt of: the eighth grade teachers spend more class time on four of the 
r^iqhc listed problem solving activities. Nevertheless, two-thirds do 
give more attention to learning problem solving techniques. 

The changes are similar for mathematical communication. Eighty- 
nine percent of fourth grade teachers and 77 percent of eighth grade 
teachers are placing more emphasis on writing about math. (see Table 6) 
Over 70 percent of fourth grade teachers say they are devoting more time 



■ The vast majority of teachers started using portfolios in 1991- 
92. However, approximately 2.0 percent of the teachers participated in 
the portfolio pilot the previous year. These teacher were comparing the 
present year to the year prior to 1990-91. 
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to explaining solutions to problems and discussing mathematics. But a 
substantially smaller percentage o£ eighth grade teachers report such 
increases. A majority of eighth grade teachers spend the same or less 
time in four of the five areas of mathematical communication listed on 
the survey. 



There also have been changes in the types of instructional 
activities, although this has occurred less widely than changes in 
curricula* focus. Just over two-thirds of teachers said that the 
portfolio assessment has moderately or greatly encouraged them to be 
innovative in planning mathematics lessons and activities, A slight 
majority of fourth grade teachers engage in more open-ended activities 
and activities involving novel materials or supplies; but less than onp- 
ho if of eiahth grade teachers do so. (see Table 7) 



Tht- portfolio assessment also has affected the organisation of 
::.,r ih^^. :'•<; instruction. About one-half of fourth grade teachers and 

* hi i d c>i the ^iqhth grade teachers have changed the way they group 
student's &m ing class in ways consistent with portfolio program 
ob^ct.i vos . There has been a modest shift away from individual work and 
toward whole class discussion, mixed ability groups, and working in 
pairs. However, for each type of class grouping a substantial 
proportion of the teachers at both grades reported no change. (see 
Table 8) 



Increased attention to the topics and activities encouraged by the 
math portfolio program has come at a cost to other areas of the 
mathematics curriculum and, at grade four, to other subjects. Two- 
thirds of teachers are choosing to spend less time on computational 



Insert Table 6 about here 



Ins -i-t Table 7 about here 



Insert Table 8 about here 
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skills and "other traditional math topics," and the majority of 
teachers agreed with the statement that "the portfolio assessment makes 
it more difficult t;o cover the mathematics curriculum." (see Table 9) 
'in the fourth grade, 44 percent of teachers are spending less time on 
subjects ot-her than math and writing. 



Insert Table 9 about here 



We asked teachers to rate the frequency with which various 
classroom activities occurred. Problem solving activities of one sort 
or another occur, on average, once per week. (see Table 10) Although 
problem solving occurs less frequently than computation (which takes 
place- two to three times per week) , teachers indicate they are doing 
.ror.siderably more problem solving now than prior to the introduction of 
-he portfolios. More unusual and challenging problem solving activities 
occu i 1 »:■ s s oft en . 



Insert Table 10 about nere 



Teachers expressed concern that "basic skills" are getting lost in 
\hu portfolio effort. In their written comments they frequently noted 
thriV p<ut folio activities take time away from basic skills and 
computation, which still need attention. One of the most common open- 
prided comments was about the difficulty of finding time for the normal 
math, .■\;rr:cul , jm and portfolios. As one teacher stated, "Urtil the 
cur: icuium outlines change to allow more port fol io- 1 ike tasks teachers 
will be doing a balancing act between covering the curriculum and 
embracing portfolio tasks." For many teachers math portfolios are 
another add-on to an already busy curriculum, forcing them to make 
di f f icuit choi ces . 

Student Performance 

Teachers are evenly split in their opinions about whether the 
program is promoting greater learning of mathematics. Fifty-one percent 
report that students are learning mathematics better because of the 
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portfolios, while 40 percent believe student learning is "Neither better 
nor worse." Only 9 percent feel that portfolios have actually been 
detrimental to students. 4 We asked teachers to explain their responses 
to this item and 77 percent did so, often in considerable detail 
Positive statements about student learning (made in 69 percent of the 
comments) focus mainly on improvements in students' thinking and 
reasoning about math. Also common are comments that portfolios 
encourage students to explain their ideas and relate math tc real life, 
which improves their understanding of mathematical concepts. 

Over one-half of the teachers made negative comments about the 
impact cf portfolios on student learning, often mentioning that learning 
is worse (or not any better) because other areas of the math curriculum 
have tc be cut to make time for portfolios. The most frequent negative 
teacher remark (made by 15% of the teachers who commented on this item) 
is a reference to cutting back on basic skills or computation. Many 
feel the need for better balance between these activities and 
portfolios. Another frequent concern is that younger students are being 
turned-off to math because of the writing demands of portfolio tasks. 
Several fourth grade teachers (13% of the student learning comments) 
mentioned that the writing required for math portfolio tasks is 
developmental ly inappropriate, particularly writing that relates to the 
PS 4 criterion/ 1 Teachers also repeatedly expressed the need for 
portfolios to be implemented at all grade levels for there to be a 
significant impact on student learning. Three-quarters of teachers at 
grade 4 and two-thirds at grade 8 agree with the statement that "Math 
portfolios should be expanded to all students in all grades." 

Differences between students' performance on traditional 
mathematics assignments and portfolio tasks was less this year than in 
1991-92. On average teachers said one-half of their students performed 

' ; These results are based on teachers' professional judgment about 
student learning and student ability levels. They should be interpreted 
cautiously since student learning may be difficult to characterize 
across ability levels, especially in the midst of a substantial shift in 
curricu lum focus and instructional practice . 

r - PS4 : What decisions, finding, conclusions, observations, 
connections and generalizations the student reached? 
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about; the same on the two types of tasks, compared to about one-third of 
the students in the previous evaluation. Nevertheless, about one-third 
of fourth grade students and one-quarter of eighth grade students did 
worse on portfolio tasks than on traditional math assignments, while the 
remaindei did better. 

There does appear to be a correlation between students' ability 
levels and how well they respond to portfolio work. Teachers generally 
think that high ability students have a more positive reaction to 
portfolios than do low and average ability students. For example, high 
ability students are more likely to "enjoy portfolio work more than 
regular math assignments," and are less likely to be hampered on math 
portfolio tasks because of poor writing skills. (see Table 11) 
Teachers indicate that a smaller proportion of their low ability 
students are "learning more math because of portfolios." And while most 
teachers report that few or none of their students find portfolio 
problems easier than traditional assignments, they find this to be true 
least often with low ability students. 



Teacher Attitudes Toward the Portfolios 

Teachers have mixed views about the mathematics portfolio program. 
Although there is broad support for portfolios, there is also 
substantial concern about the implementation of the program and about 
specific uses of portfolios. Teachers' written comments reflect a mix 
of enthusiasm and frustration over portfolios. Statements of support 
for the philosophy behind portfolios are often followed by concerns 
about state demands. 

Teachers think the portfolios are helpful as informal classroom 
assessment tools, but worry about their use for external assessment 
purposes. The majority agree or strongly agree that the portfolios help 
students monitor their own progress, and that portfolios are useful for 
informing parents about student progress. The majority also agree that 
portfolio scores should be used as part of students' grades, although 



Insert Table 11 about here 
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about one-half of the teachers judge students math work differently when 
assigning grades than when scoring for the portfolios. Most find the 
portfolio criteria easy to use, but about one-third report frequent 
difficulty applying crit :ia PS3, PS4 and CI. (see Table 12) 



In contrast, teachers are more cautious about the use of portfolios 
for external assessment purposes. The vast majority of teachers do not 
believe it would be fair to evaluate them on the basis of students' 
portfolio scores. While the majority of fourth grade teachers think 
portfolio scores are a better measure of math learning than standardized 
tests, eighth grade teachers are about evenly divided between those who 
agree, those who disagree and those who are uncertain. One of the most 
common concerns raised by teachers in their open-ended comments was the 
state's strong emphasis on scoring. Many feel that the emphasis on 
reliable scoring is misguided and it perverts the original purpose of 
portfolios as a tool for assessing an individual student's growth. One 
teacher noted that "The state wants portfolios to be scored like a 
bubble test for their own purposes .. .Theoretically, the portfolios were 
show personal growth in math and writing abilities." 

Teachers are concerned about the validity of portfolios as an 
a s percent instrument. Expressing a common sentiment one teacher asked 
"How car: the validity of scoring outside the classroom be justified when 
"here are so many uncontrolled variables?" About one out of every four 
[I'M) teachers who commented on "other issues" expressed concerns along 
these lines. Several teachers (5%) also worried that some of their 
colleagues were providing students with opportunities to improve their 
work that they felt were inappropriate and were not permitted in their 
own classes. 

There is a strong sense that the changes brought about by 
portfolios have had a positive impact on mathematics education. Many 
teachers expressed some support for the portfolio philosophy along with 
their complaints about the program. For example, fifty-four percent of 



Insert Table 12 about here 
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teachers agreed with the statement that "the Vermont mathematics 
portfolio assessment is moving education in the right direction," while 
only 21 percent disagreed. Seventy-five percent of the teachers also 
supported expanding the portfolio assessment to all students in all 
grades . 

Time Burdens 

As in the past, time burdens are teachers' greatest concern; 
portfolios consume considerable time both in class and outside of class. 
However, estimates of the exact amount of time devoted to portfolios are 
somewhat elusive. Teachers' relative and absolute judgments about the 
time demands of portfolios are somewhat contradictory. On the one hand, 
55 percent of teachers who used portfolios in 1991-92 are spending more 
out-of-class time this year on portfolios than they did last year, and 
60 percent are spending more classroom time than in 1991-92. Fewer than 
10 percent report decreases in either figure. On the other hand,, 
estimates of the actual number of hours spent in these two types of 
activities have gone down by approximately one-third from 1991-92. The 
difference in these estimates may be explained.- in part, by changes in 
the way the absolute time estimates were gathered between the two 
years . ' 

Other responses support the conclusion that the portfolios continue 
to make significant demands on teachers. For example, most teachers at 
both grade 4 and 8 feel that they spend too much time managing and 
scoring portfolios. (see Table 13) Most teachers do not feel that the 
demands of the mathematics portfolio program are lessening, and many are 
displeased that the burden continues to be so great. Less than one- 

' In addition to asking teachers to compare this year to last year 
we also asked them to estimate the actual number of hours they spent on 
selected portfolio related activities. There were inconsistencies 
between the results reported in 1991-92 and 1992-93. Although teachers 
reported spending more time in 1992-93 than previously, their estimates 
of hours were lower than in 1991-92. Differences in format between the 
1992-93 and l°91-92 questionnaires may have contributed to this 
inconsistency. Since the vast majority of teachers reported spending 
the same or more time this year than last, we place more confidence in 
the relative judgments than m the hour estimates. 
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third of the teachers agreed with the statement that "Overall, 
portfolios are less of a burden on me this year than last year." 



Insert Table 13 about here 



Many observers expected that the demands placed on teachers by the 
portfolio assessment program would diminish as teachers became more 
experienced, and this was true to a limited degree. Experience yielded 
benefits for some teachers: forty to fifty percent of the teachers who 
participated in the program in 1991-92 said that specific portfolio 
related activities, such as finding interesting tasks and teaching 
problem solving, had oecome easier. However, in most cases a similar 
proportion of teachers said that the activity was no easier than the 
year before, (see Table 14) Simila? ly, we could find little evidence 
that teachers with three years of portfolio experience found portfolios 
easier or less time consuming than teachers in their second year. Third 
year portfolio users were slightly less likely than second year 
portfolio users to find managing portfolios too time consuming (54% to 
4~%) ; but this was the only significant difference. 



Insert Table 14 about here 



One of the most common issues raised by teachers in their open- 
ended comments (mentioned by 25% of those who commented on "other 
issues") was the excessive burden placed on fourth and eighth grade 
students and teachers by the combination of writing and mathematics 
portfolios. They feel they spend too much time away from their students 
tor training, and that they carry a' burden that is not placed on 
teachers and students at other grade levels. One teacher warned "I 
support portfolios but fear that you'll lose your allies (even me) by 
o ve r wo r k i ng t h em ! " 
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Summary 

Overall, the message from Vermont teachers about the portfolio 
assessment system remains mixed. Most teachers have modified their 
curricula and teaching practices to emphasize problem solving and 
mathematical communication skills, but many feel they are doing this at 
the expense of other areas of the curriculum, especially basic skills 
and computation. Most teachers express support for mathematics 
portfolios in a general sense, but there are widespread concerns about 
using portfolios as an external evaluation tool and, most of all, about 
the time demands of planning, administering, and scoring portfolio 
problems. There is little evidence that the burdens on teachers are 
lessening significantly over time. Furthermore, it is clear that 
variations in teachers' approaches to implementing mathematics 
portfolios persist. 

Teachers' responses suggest some ways Vermont might improve the 
portfolio assessment system in the future. First, teachers express 
strong support for expanding portfolios to all grade levels. For many 
teachers this is point of efficiency as well as fairness. As long as 
portfolios are limited to grades 4 and 8, other grades will be slow to 
adopt practices that support the skills emphasized by the portfolio 
assessment system. Also, parents, administrators, and other teachers 
will continue to expect all teachers at grades 4 and 3 to abide by the 
"traditional curriculum." Secondly, the developmental appropriateness 
of certain aspects of the grade 4 mathematics portfolios should be re- 
examined. Many fourth grade teachers are convinced that the writing 
demands are too great for many of their students. Finally, the State 
Department of Education should review the balance between local 
flexibility and standardization of implementation -- while flexibility 
contributes to the mcraningf ulness of the portfolios as local 
instructional tools, it reduces the validity of inferences that can be 
drawn' from the portfolio scores. 
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Table 1 



CHARACTERISTICS 


OF SAMPLED 


TEACHERS , 


1991-92 


AND 


1992-93 




1991- 


92 




1992 


-93 




Grade 4 


Grade 8 


Grade 


4 


Grade 8 


Number 


112 


32 


382 




137 


Response Rate 


90% 


67% 


52% 




41% 


Mean Years 
Experience 


15.0 


16.7 


14.6 




16.2 


Percent Special- 
izina in math 


4.7% 


73.0% 


1.6% 




69.3% 
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Table 2 

ASSISTANCE ALLOWED BY TEACHERS ON BEST PIECES 
(Percentage of Teachers) 



Allowed to assist on Rules 

which best pieces? Dif fer for 

Source Grade None Some Most All Each 

Student 



The teacher 4 27 23 14 16 21 

8 27 32 9 13 19 

Other students 4 34 31 11 12 11 

8 23 39 11 12 15 

Parents or others 4 * 71 13 4 4 8 

outside of school 8 39 28 8 13 11 



NOTE: * Grade level difference significant at the 5% level (p-. .05) 



20 
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Table 3 

WHO SELECTS BEST PIECES 
(Percentage of Teachers) 



Who selects best pieces? Grade 4* Grade 8 

Students on their own 21 30 
Students with limited 

teacher input 55 57 
Students and teachers have 

equal role 18 8 
Teacher with limited student 

input 5 3 

Teacher I 1 

NOTE: * Grade level difference significant at the 
5% level (p<.05) 
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Table 4 

TEACHER" EMPHASIS ON PORTFOLIO CHARACTERISTICS 
(Percentage of Teachers J 



Area of Emphasis 






Amount 


or onpnasis 




Grade 


None 


Minor 


Moderate 


Heavy 


Mathematically correct 


4 


5 


32 


54 


10 




8 


8 


21 


58 


13 


Neat and polished 


4 


6 


40 


49 


6 


aooea ranee 


8 


Q 


38 


49 


4 


Interesting or important 


4 * 


1 


6 


52 


41 


to students 


8 


2 


14 


58 


26 


Similar to examples in 


4 


13 


29 


45 


13 


Resource Guide 


8 


17 


32 


42 


9 


Similar to good examples 


4 


6 


22 


48 


24 


from scoring training 


8 


10 


25 


46 


19 


Related to problem 


4 


1 


6 


42 


51 


solving criteria 


8 


2 


9 


48 


40 


Related to mathematical 


4 


2 


10 


47 


41 


communication criteria 


8 


2 


17 


47 


34 



NOTE: * Grade level difference significant at the 5% level (p<.05) 
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Table 5 

CHANGE IN TIME SPENT ON PROBLEM SOLVING ACTIVITIES 
(Percentage of Teachers) 



Activity 


Grade 


Somewhat or 
Much Less 


About the 
Same 


Somewhat or 
Much More 


Exploring Patterns 


4 


4 


42 


54 




8 


5 


57 


38 


Applying math knowledge 


4 * 


22 


34 


44 


to traditional word 


8 


17 


56 


2S 


nrnhl ottiq 










Applying math knowledge 


4 * 


2 


23 


75 




8 


1 


29 


70 


Solving logic or reason- 


4 * 


1 


24 


75 




8 


5 


43 


51 


Applying math to 










problems in a real world 


4 * 


2 


26 


71 


setting 


8 


3 


43 


54 


Collecting and analyzing 


4 * 


3 


38 


59 


data 


8 


10 


45 


44 


Learning problem solving 


4 * 


1 


16 


83 


techniques 


8 


3 


3 4 


63 


Exam i n i ng incorrect 


4 


5 


45 


50 


solutions 


8 


8 


54 


38 



MOTE: * Grade level difference significant at the 5% level (p<.05) 
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Table 6 

CHANGE IN TIME SPENT ON MATHEMATICAL COMMUNICATION 
(Percentage of Teachers) 



Activity 


S 

Grade 


omewhat or 
Much Less 


About the 
Same 


Somewhat or 
Much More 


Writing about mathe- 


4 * 


3 


8 


89 " 


matics 


8 


7 


16 


77 


Explaining solutions to 


4 * 


3 


25 


72 


problems 


8 


15 


53 


32 


Discussing mathematics 


4 * 


1 


29 


71 




8 


5 


57 ' 


37 


Making or interpreting 


4 * 


1 


29 


70 


charts, graphs, diagrams 8 


7 


45 


49 


Writing reports about 


4 


5 


44 


51 


mathematics 


8 


9 


41 


50 


Describing feelings 


4 * 


6 


51 


4 3 


about mathematics 


8 


11 


60 


30 


NOTE: * Grade level 


di f f erence 


signify cant 


at the 5% 


level (p<.05) 
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Table 7 

CHANGE IN CliASSROOM ACTIVITIES 
(Percentage of Teachers) 



Act ivity 


Grade 


Somewhat or 
Much Less 


About the 
Same 


Somewhat or 
Much More 


Assign activities whose 


4 


3 


29 


68 


outcome and /or duration 


8 


2 


36 


63 


is uncertain 










Vary schedule or length 


4 * 


1 


27 


72 


of math activities 


8 


3 


54 


42 


Involve students in 


4 


3 


46 


51 


hands-on math activities 


8 


3 


57 


38 


Use supplemental math 


4 


14 


50 


36 


books 


8 


1.0 


60 


30 


Use novel materials or 


4 


2 


46 


52 


supplies in math lessons 


8 


3 


50 


47 



NOTE: * Grade level difference significant at the 5% level (p<,05) 
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Table 8 

CHANGE IN CLASSROOM ORGANIZATION 
(Percentage of Teachers) 

Somewhat or About the Somewhat or 
Act ivity Grade Much Less Same Much More 



Discussing together as a 4 * 
whole class 8 

Working in groups with 4 
students of similar 8 
abi lity *' 

Working in groups with 4 
students of different 8 
abilities 

Working in pairs 4 

8 

Working individually 4 

8 



3 39 58 

14 58 28 

19 59 22 

18 59 23 

2 52 4 6 

2 62 36 

2 54 44 

3 60 3 7 

30 58 12 

32 64 3 



!OTE: * Grade level difference significant at the 5% level (p--.0b) 
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Table 9 



CHANGES IN THE ALLOCATION OF CLASS TIME 
(Percentage of Teachers) 



Activity 


Grade 


Somewhat or 
Much Less 


About the 
Same 


Somewhat or 
Much More 


Any math activity 


4 * 


15 


39 


46 




8 


28 


52 


20 


Computat ion 


4 * 


65 


30 


5 




8 


54 


43 


3 


Other traditional math 


4 


63 


32 


5 


topics 


8 


56 


42 


3 


Any writing activity 


4 


6 


31 


63 




8 


5 


24 


71 


Subjects other than math 


4 * 


44 


47 


9 


and writing 


8 


9 


71 


20 



NOTE: * Grade level difference significant at the 5% level ip<,05) 



Table 10 



FREQUENCY OF CLASS ENGAGEMENT IN VARIOUS MATHEMATICS ACTIVITIES 

(Percentage of Teachers) 

1-3/ 1-3 Once 2-3/ 
Activity Grade Never per pei per per Daily 
Sem. Month Week Week 

Computation and other 4 0 1 2 9 60 28 

traditional math topics 8 0 2 6 12 51 29 

Writing about 4*2 9 21 41 24 3 

mathematics 8 5 15 26 40 ' 11 3 

Applying math knowledge 4 1 5 18 38 33 5 

to solve novel problems 8 2 9 35 33 17 4 

Learning problem-solving 4 * 0 1 15 44 34 6 

techniques 8 1 5 27 28 29 11 

Explaining solutions to 4 * 0 2 11 41 34 11 

problems 8 0 2 14 32 28 24 

Working in groups with 4 * i 4 12 22 31 30 

students of different 8 5 8 21 18 24 24 
abilities 

Working on activities 4*3 9 25 37 20 7 

whose outcome and/or 8 4 17 32 3 3 13 3 
duration is unknown 

Using novel materials or 4 * 3 11 28 26 25 9 

supplies in math lessons 8 3 22 27 29 12 7 

NOTE: * Grade level difference significant at the 5% level (p<.05) 
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Table 11 

STUDENT REACTIONS TO MATHEMATICS PORTFOLIOS BY GRADE AND ABILITY LEVEL 

(Percentage of Teachers) 



Student Reactions 


Grade 


Percent of Teachers Reporting Most /Almost 

All 


Low Ability 
Students 


Ave. Ability 
Students 


High Ability 
Students 


Enjoy doing portfolio tasks 


4 - 


10 


15 


43 


more than regular math 


8 


18 


23 


38 


assignments 










Like portfolios better this 


4 


14 


21 


36 


year than last year 


8 


17 


23 


28 


Learn more math because of 


4 


21 


30 


49 


the portfolios 


8 


25 


28 


32 


Find portfolio tasks easier. 


4 


4 


6 


14 


than traditional 


8 


10 


10 


17 


assignments 










Portfolio tasks do not re- 


4 


46 


5 


3 


flect math ability because 


8 


31 


6 


2 


or poor writing skills 
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Table 12 



DIFFICULTY APPLYING SCORING CRITERIA TO STUDENT PORTFOLIOS 

(Percent of Teachers) 



Criteria 



PS1 Understanding 



Grade 



4 
8 



Never or 
Seldom 



76 
73 



Occasion- 
ally^ 



20 
24 



Often or 
Verv Ofter 



52 
51 



38 
43 



10 
6 



PS 3 Why": 



4 
8 



24 
27 



48 
47 



28 
26 



PS 4 what? 



30 
35 



32 
25 



38 
40 



CI Language 



4 
8 



32 
32 



40 
38 



28 
30 



C2 Reoresentat ions 



44 
47 



41 
41 



15 
12 



C3 Presentation 



4 

8 



43 
41 



40 
47 



16 
12 



? 
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Table 13 

DEMANDS OF THE MATHEMATICS PORTFOLIO PROGRAM ON TEACHERS 
(Percentage of Teachers) 



Statement 


Grade 


Disagree 


Neutral 


Agree 


It is easy to prepare 


4 


60 


22 


18 


portfolio lessons 


8 


52 


30 


18 


I spend too much time 


4 


21 


26 


53 


managing portfolios 


8 


22 


28 


49 


Over a 1 1 , portfolios are 


4 


55 


16 


29 


less of a burden on me 


8 


52 


25 


22 - 


this year than last year 










Scoring portfolio work 


4 


84 


7 


9 


is not' too time 


8 


83 


7 


10 


consuming 










The portfolio assessment 


4 


21 


15 


65 


makes it more difficult 


8 


16 


20 


64 


to cover the mathematics 










curriculum 










NOTE: * Grade level difference 


significant 


at the 5% 


level (p<.05) 
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Table 14 



CHANGE IN DIFFICULTY OF 


PORTFOLIO RELATED 


TEACHER 


ACTIVITIES 


(Percentage 


of Teachers) 










More About the 




Activity 


Grade 


Hi f f i nil f 

Ull 1. 1LU1 L. 


Same 


Easier 


Find interesting tasks 


4 


10 


41 


49 




8 


1 1 


A 1 


36 


Decide if task is 


4 


9 


35 


56 


appropriate for 


8 


1 D 


_> z. 


53 


port loi. io assessment 










Integrate tasks into the 


4 


1 J 




43 


maun cur r iluiuiu 


Q 

o 


21 


48 


32 


Teaching problem solving 


4 


1 u 


7 Q 

J .7 


51 




« 
o 


6 


48 


4 6 

-i U 


lCCll_.iii.liy illC* L. 1 iC-lllCi L. X *_C* J. 


4 


16 


46 


39 


communicat ion 


8 


14 


47 


40 


Motivate students to 


4 


18 


52 


30 


work on portfolio tasks 


8 


26 


46 


28 


Make students understand 


4 


18 


41 


42 


qualities of good pieces 


8 


19 


34 


47 


Explain the portfolios 


4 


9 


56 


36 


to parents 


8 


6 


68 


26 



NOTE: * Grade level difference significant at the 5% level (p<.05) 
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